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This work is an attempt to improve the Bayesian neural network (BNN) for studying photoneutron yield cross 
sections as a function of the charge number Z, mass number A, and incident energy £. The BNN was improved 
in terms of three aspects: numerical parameters, input layer, and network structure. First, by minimizing the 
deviations between the predictions and data, the numerical parameters, including the hidden layer number, 
hidden node number, and activation function, were selected. It was found that the BNN with three hidden 
layers, 10 hidden nodes, and sigmoid activation function provided the smallest deviations. Second, based on 
known knowledge, such as the isospin dependence and shape effect, the optimal ground-state properties were 
selected as input neurons. Third, the Lorentzian function was applied to map the hidden nodes to the output 
cross sections, and the empirical formula of the Lorentzian parameters was applied to link some of the input 
nodes to the output cross sections. It was found that the last two aspects improved the predictions and avoided 


overfitting, especially for the axially deformed nucleus. 


Keywords: Bayesian neural network, Photoneutron cross sections 


I. INTRODUCTION 


Neural networks are powerful tools for making predictions 
after training using data and have made possible exciting 
achievements in nuclear physics in the past few years [1— 
6]. The earliest work on neural networks in nuclear physics 
dates back to 1993, when the phenomenological approach to 
many-body systems based on multilayer feedforward neural 
networks was introduced to learn the systematics of atomic 
masses and nuclear spins and parities [7]. Thereafter, various 
types of neural networks have been applied to study nuclear 
mass systematics [8, 9], G-decay systematics [10], and bind- 
ing energy [11]. 

Recently, with the help of physical ideas, neural net- 
works have been improved and their potential capability has 
been realized. The known physics were explicitly embed- 
ded based on the Bayesian neural network (BNN), which 
results in a novel method for accurately predicting G-decay 
half-lives[12]. The input data are preprocessed, correlations 
among the input data are included, and the problem of mul- 
tiple solutions can be reduced, yielding more stable extrap- 
olated results[13]. The combination of the three-parameter 
formula and the BNN results in a novel approach for describ- 
ing the charge radii of the nuclei [14]. In addition, the physi- 
cal law is revealed by neural networks. For example, a feed- 
forward neural network model was trained to calculate nu- 
clear charge radii, and the correlation between the symmetry 
energy and charge radii of Ca isotopes was suggested [15]. 
The convolutional neural network algorithm was applied to 
determine the impact parameters in heavy ion collisions us- 
ing constrained molecular-dynamics model simulations [16]. 
To date, neural networks have been widely applied for sig- 
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nal identification [17—19], data restoration [20, 21], and re- 
gression analysis [5] and have been broadly used in nuclear 
physics. 

In studies where the nuclear masses and nuclear charge 
radii were studied, Utama et al. claimed that the physics in 
the initial prediction can be included using physics-motivated 
models and that the BNN can be used to fine-tune these mod- 
els by modeling the residuals [22, 23]. Based on this residual- 
approach, the predictions of several physics-motivated mod- 
els in nuclear physics have been improved using neural net- 
works. For example, the BNN approach was employed to im- 
prove the nuclear mass predictions of various physics models 
[24] and the fission yield prediction using the TALYS model 
[2]. To study the isotopic cross sections in proton-induced 
spallation reactions, the BNN predictions obtained by learn- 
ing the experimental data and the residuals of the SPACS 
parametrization were compared [25]. It was shown that the 
latter are better, which indicates the significance of the robust 
physics-motivated model when using a neural network. 


A neural network is a type of numerical algorithm. In 
cases where a physics-motivated model is not available, at- 
tempts have been made to provide physics guidance in neural 
networks. A multitask neural network was applied to learn 
the experimental data of the giant dipole resonance parame- 
ters directly, and the predictions were better than those cal- 
culated by the Goldhaber—Teller model [26]. In a study for 
determininig the impact parameters of heavy-ion collisions 
using convolutional neural networks, no initial predictions 
were made [16]. In addition to the physics-motivated model, 
physics guidance in neural networks has also been provided 
from the input layer [27] or by an empirical formula [14]. 
In this study, an attempt is made to improve the BNN for 
studying photoneutron yield cross sections, where both the 
improvements from the input layer and by the empirical for- 
mula are considered. 


Photonuclear reactions were first observed more than 60 
years ago [28]. Their cross section data are important to ana- 
lyze the radiation transport, study the nuclear waste transmu- 


tation [29], and calculate the nucleosynthesis [30, 31]. The 
underlying mechanisms in photonuclear reactions are signif- 
icant in fundamental nuclear physics [32, 33]. More than 
27000 data of Cyn were collected in the EXFOR database 
[34] for nuclei from ĉLi to 2°°Pu at incident energies above 
the neutron separation energy. New facilities have been de- 
veloped to measure additional data [35, 36]. A large dataset 
makes machine learning possible and advisable. 

This study focuses on the improvement of the BNN to 
study photoneutron yield cross sections. The remainder of 
this paper is organized as follows. In Sect. II, the model is 
described. In Sect. III, both the results and discussions are 
presented. Finally, a summary is presented in Sect. IV. 


Il. MODEL 


The fundamentals of the BNN approach were established 
in the last century [37]. It is now a commonly used method 
for pattern recognition and numerical regression. The latter is 
the case in the present study. In the normal BNN algorithm, a 
neural network with hidden layers is built for mapping from 
the input layer X to the output layer Y. 
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where 0 = {a,bj,cj, dji} are parameters in the neural net- 
work, {a, cj} are biases, and {b}, dji} are weights. The ac- 
tivation function f can be set as a sigmoid, tanh, or softplus 
type. 
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With the activation function, nonlinearity between the input 
and output is realized. 

The parameters in Eq. (1) are determined by learning 
dataset D, which displays the outputs Y for given inputs X, 
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where Ng is the sample size of the available data. The 
Bayesian theorem, which deduces the posterior knowledge 
from the prior case using the likelihood function, is applied 
to solve this regression problem. Specifically, the prior distri- 
bution P(@) of the parameters 0 is assumed, and its posterior 
distribution P(@|D) for a given dataset D is expressed as 
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where P(D|@) is the likelihood of D given 8. 


With the posterior distribution of the parameters 0, the ex- 
pected value of the output Y* for the known inputs X* is 
expressed as the integration 


Y*= J YOPE), (5) 


Monte Carlo techniques are applied to calculate the above in- 
tegration. 


Y* = MC Spion) [Y (X*,0)] 
aL (6) 


where MCS pojn) denotes the Monte Carlo sampling drawn 
from the posterior distributions P(0|D) and 0) (k = 
1,2,..., Ns) is the k-th sample drawn from the posterior dis- 
tribution P(@|D) with total sample number N,. The accuracy 
of Eq. (6) to evaluate Eq. (5) is expressed as 
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where S is the standard deviation of the samples Y (X*, 0°*)), 
The difference between the integration and Monte Carlo cal- 
culations can be reduced by increasing the number of samples 
Ns. 

The analytic computation of the posterior distribution 
P(6|D) is intractable owing to the high dimensionality of the 
parameters. In the BNN approach, variation inference is ap- 
plied to obtain an approximation of P(@|D). The variation 
inference attempts to find a « such that q(0|%) is the minimum 
distance from P(6|D) measured by the (KL) divergence: 
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The traditional method of solving the regression problem 
in physics is based on an empirical formula with parameters, 
where an appropriate formula can avoid both misconvergence 
and overfitting. Similarly, when applying a BNN in physics, 
it is important to select the appropriate input nodes for a spe- 
cific output. In this study, the output is the photoneutron yield 
cross section Czn. More than 27000 data of Cyn were col- 
lected in the EXFOR [34] for nuclei from ĉLi to ?3°Pu at 
incident energies above the neutron separation energy. The 
minimum input nodes of the BNN to study Cyn are the charge 
number Z of the target, mass number A of the target, and en- 
ergy £ of the incident y particle. The BNN in this case is 
abbreviated as BNN-ZAE and illustrated in Fig. 1(a). 
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Fig. 1. (Color online) (a) Bayesian neural network BNN-ZAE with 
input nodes of charge number Z, mass number A, and incident en- 
ergy £. (b) Bayesian neural network BNN-OPT with optimal in- 
put nodes including mass number A, incident energy £, and other 
ground-state properties. (c) Lorentzian function-based Bayesian 
neural network LBNN where the Lorentzian shape of the photoneu- 
tron cross sections and the empirical formula of the Lorentzian pa- 
rameters are considered. 


The BNN is a numerical algorithm. In physics, the BNN is 
used to learn the residuals of the physics-motivated model and 
then fine-tune the model. Thus, the main physics information 
is included in the initial prediction by the physics-motivated 
model. In our previous work [38], we illustrated a new 
method to provide physical guidance in the BNN from the in- 
put layer without initial prediction by the physics-motivated 
model. This method is applied in this work to select the op- 
timal ground-state properties as neurons of the input layer in 
the BNN to predict the photoneutron yield cross sections ogn. 
Details of the method are provided in Ref. [38]. In brief, vari- 
ous combinations of ground-state properties are considered as 
the input nodes in the BNN, and the optimal combination is 
selected according to the smallest deviation between the data 
and the prediction. The optimal input nodes are as follows: 


X= {Sn, Qa, B, A, Bo,e}, (9) 


where Sh is the neutron separation energy, Qg is the 87 de- 
cay energy, B is the binding energy per nucleon, A is the mass 
number, 62 is the quadrupole deformation parameter, and € is 
the incident photon energy. In this case, the BNN is abbre- 
viated as BNN-OPT and illustrated in Fig. 1(b). It is worth 
noting that the charge number Z is not input in the BNN-OPT 
model. Only the experimental data of the photoneutron yield 
cross section for stable nuclei are available. For stable nuclei, 
the correlation between the charge number Z and the mass 
number A is very strong (see Fig. 1 in Ref. [38]). After mass 
number A has been used, the additional Z is not conducive to 


reducing the deviation between the data and the prediction by 
the BNN-OPT model. 

We will further prove that the BNN can be improved by 
considering known knowledge of physics. The photoneutron 
yield cross sections oz, as a function of incident energy can 
be characterized by a Lorentzian shape with two components. 
The Lorentzian parameters are peak energy L;, width [;, and 
strength s;. 
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The subscripts 7 = 1 and 2 denote the two components. 
OTRK expresses the cross section in terms of the Thomas— 
Reiche—Kuhn sum rule. The Lorentzian function is applied 
before mapping the hidden nodes to the output cross sec- 
tions, as shown in Fig. 1(c). Except for the Lorentzian shape, 
the known knowledge includes the empirical formula of the 
Lorentzian parameters: 


E, = HA"? — Hofo, 
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where H; (i= 1, ..., 6) are empirical parameters. These empir- 
ical formulas are also considered in the BNN. The algorithm 
then becomes a Lorentzian function-based BNN (LBNN). As 
shown in Fig. 1(c), the (black) solid lines show that all in- 
puts nodes are used to calculate the empirical parameters H1, 
Hə, H3, H4, Hs, and Hg. Then, the empirical parameters, to- 
gether with the input nodes A and (bg, are used to calculate the 
Lorentzian parameters E1, E2, I1, I2, S1, and s2 according 
to Eq. (11). As can be observed, the Lorentzian parameters 
depend not only on A and 2 but also on other input nodes. 
However, the dependence on A and 62 is known and phys- 
ical, as shown in Eq. (11), whereas the dependence on all 
the input nodes is unknown and numerical. These were fitted 
during the training process. We use two types of lines, solid 
and dashed, to distinguish between the physical and numeri- 
cal dependences. Energy is one of the input nodes, but is not 
used to calculate the hidden layer. This is because the depen- 
dence of the energy on the output (cross section) is known as 
the Lorentzian shape. In the BNN-ZAE and BNN-OPT mod- 
els, the dependence on energy of the output (cross section) 
is a black box. In the LBNN model, some dependencies on 
the inputs of the outputs are known. We say that parts of the 
black box are open. 


I. RESULTS & DISCUSSIONS 


There are three types of BNN in this work: BNN-ZAE, 
BNN-OPT, and LBNN. In the latter two, physics guidance is 
provided by improving the input nodes and considering the 


Lorentzian shape. In contrast, BNN-ZAE is a numerical al- 
gorithm without any physical improvement. In the follow- 
ing section, we evaluate these three models by comparing 
their predictions. The root-mean-square (RMS) deviations 
between the predictions and data were calculated as 


Š (n) (n)\? 
RMS = Wy (lose —log oS is 
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(12) 


where oy is the predicted cross section and oq is the corre- 
sponding data. Log-scaling was used because the values of 
Oa are in the region from 107° to 10 b, which is across four 
orders of magnitude. 
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Fig. 2. (Color online) Root-mean-square (RM S) deviations be- 
tween data and the BNN-ZAE predictions as a function of the it- 
eration step. The insets show the RM S deviations at the iteration 
step 4000. (a) Comparison of the RM S deviations for 1, 2, and 3 
hidden layers. (b) Comparison of the RMS deviations for 1 hid- 
den layer with 10, 30, and 100 hidden nodes. (c) Comparison of the 
RMS deviations for 1 hidden layer with 30 hidden nodes but differ- 
ent activation functions. 


The RMS deviations as a function of the iteration step 
were applied to test the convergence of the predictions. The 
cases for the BNN-ZAE predictions with one, two, and three 
hidden layers and 10 nodes for each hidden layer are shown in 
Fig. 2(a). From the figure, we can observe that all the RMS 
deviations converge quickly in 1000 iteration steps. The final 
RMS deviations were 0.226, 0.219, and 0.214 for the cases 
of one, two, and three hidden layers, respectively. This in- 
dicates that more hidden layers help reproduce the training 
data; however, the effect is weak. The effect of the number of 
hidden nodes was also tested by setting one hidden layer to 
10, 30, and 100 nodes. The RM S deviations are compared 
in Fig. 2(b). It was shown that the rate of convergence was 
slower for a larger number of hidden nodes. The iteration 
steps required to decrease the RMS deviations to a value less 
than 0.25 are 1000 for the cases of 10 and 30 hidden nodes, 
but 2000 steps are needed for 100 hidden nodes. The RM S 


deviations at 4000 iteration steps are 0.226, 0.229, and 0.235 
for 10, 30, and 100 nodes, respectively. The RM S deviations 
are similar for 10 and 30 hidden nodes, but they are larger 
for 100 hidden nodes. The role of the activation function was 
tested by setting a hidden layer with 30 hidden nodes using 
the sigmoid, tanh, or softplus activation function, as shown 
in Eq. (2). The RM S deviations are compared in Fig. 2(c). 
The sigmoid activation function was shown to be the best. In 
the following calculations, three hidden layers with 30 hidden 
nodes for each layer and a sigmoid activation function were 
applied. 
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Fig. 3. (Color online) Comparison of the prediction errors for three 
types of Bayesian neural networks (as shown in Fig. 1). (a) The 
RMS deviations as a function of the iteration step. The inset shows 
the RM S deviations at the iteration step 4000. (b) Distributions of 
the differences between predictions and training data log a, — log ou. 


The RMS deviations as a function of the iteration step for 
the three types of BNNs are shown in Fig. 3(a). It is shown 
that the rates of convergence are similar for all the cases, and 
4000 iteration steps are sufficient for the calculations. The 
RMS deviations at 4000 iteration steps are 0.198 for the 
BNN-OPT model and 0.206 for the LBNN model, which are 
smaller than that of 0.214 for BNN-ZAE. Figure 2(a) and (b) 
shows that a change in the numerical parameters (number of 
hidden layers and hidden nodes) does not improve the neural 
network. Here, it is indicated that the key to improving the 
neural network for physics is the consideration of the known 
knowledge of observables. For instance, several effects of 
the photoneutron reaction, such as isospin dependence and 
shape effect, have been found [3, 39, 40], which indicate that 
the cross sections of the photoneutron reaction depend on the 
ground-state properties of the nuclei. Another known knowl- 
edge of photoneutron cross sections is their Lorentzian shape. 
The data of the ground-state properties are applied to the input 
layer of the BNN-OPT model, whereas the Lorentzian shape 
is considered in the LBNN model. These two aspects are re- 
sponsible for the smaller RM S deviations compared to those 
provided by the BNN-ZAE algorithm. 


To compare the errors of the predictions by the three neu- 
ral networks, the distributions of the differences between the 
predictions and training data logo, — log oa are shown in 
Fig. 3(b). A value 1 of log op — log og means that the pre- 
dicted cross section is 10 times larger than the experimen- 
tal data, whereas a value of -1 indicates that it is 10 times 
smaller. These cases hardly ever occur. The sample size for 
abs(log a) — log aa) > 1 is less than 0.1% of the total sam- 
ple size. For most samples, the values of log op — log og are 
in the region from -0.1 to 0.1, which means that the predic- 
tion agrees with the data in 1.26 times. Specifically, these 
samples comprise 48.0%, 57.1%, and 61.6% of BNN-ZAE, 
BNN-OPT, and LBNN samples, respectively. The percent- 
age of LBNN was the largest. Furthermore, the distribution 
of log op — log oq provided by the LBNN was more symmet- 
rical than that of the others. Considering these aspects, the 
LBNN model is superior to the BNN-OPT model. 


We further evaluated the three types of BNNs by compar- 
ing their predictions of photoneutron yield cross sections for 
spherical nuclei 9?Zr, 1!?Sn, and ?°°Pb. The cross sections 
as a function of the incident energy are shown in Fig. 4. In 
general, for spherical nuclei, only one main component of 
the Lorentzian shape is displayed in the excitation function 
of the photoneutron reaction. The experimental cross sec- 
tions as a function of the incident energy £ for nuclei 97Zr, 
112Sn, and 7°°Pb display this situation. The position and 
value of the Lorentzian peak were reproduced well by the 
LBNN model. In contrast, the peak position for nucleus °?Zr 
and the peak value for nucleus ‘!?Sn were underestimated by 
both the BNN-ZAE and BNN-OPT models. 


The data were abundant for these three nuclei. Some ex- 
perimental errors, including statistical and systematic errors, 
are shown as error bars in the figure. The error of the data was 
applied to train the BNN. More specifically, the data were re- 
sampled with a number inversely proportional to the experi- 
mental errors. This means that the data with small errors are 
resampled with a large number for training, while those with 
large errors are applied only a few times. After 4000 iteration 
steps, 100 samples are applied to calculate the standard devi- 
ations and uncertainties of the predictions, which are shown 
as shadows in the figure. In the energy region with data, the 
uncertainties of the predictions by the BNN-ZAE and BNN- 
OPT models are small, but there should be large uncertain- 
ties in the extrapolations. Because the Lorentzian function is 
applied to the neural network, the predictions and their un- 
certainties are both constrained by the Lorentzian shape. The 
uncertainties of the predictions by the LBNN model were the 
same for interpolations and extrapolations in logarithmic co- 
ordinates. Furthermore, the uncertainties are small because 
the data for these three nuclei are abundant. The uncertain- 
ties shown in Fig. 4 originate from the Monte Carlo tech- 
niques. The Lorentzian function applied in the LBNN model 
is only an approximate expression for the photoneutron yield 
cross sections. A threshold exists for the photoneutron yield 
cross sections at € = Sn, where Sh is the neutron separation 
energy. The Lorentzian shape is the most important known 
knowledge of photoneutron yield cross sections, but it does 
not consider the threshold. The experimental data near the 


threshold were against this formula. Thus, predictions by the 
LBNN model below the threshold are meaningless. 

For the axially deformed nucleus, the photoneutron yield 
cross sections as a function of incident energy display two 
main Lorentzian shapes. The difference between the two 
Lorentzian peaks has been found to be positively associated 
with the deformation parameter (2 using the time-dependent 
Hartree-Fock model [41]. The data for deformed nuclei +P, 
As, and '©°Ho are shown in Fig. 5. The quadrupole de- 
formation parameters (2 are —0.22, —0.24, and 0.28, respec- 
tively. The data clearly show two peaks for 1°°Ho, but faintly 
for As. The abundant data but large errors for 3tP make it 
difficult to distinguish the two peaks. However, the wide peak 
does not contradict the two main Lorentzian shapes because 
they may be too close to be distinguished. 

The curves and shadows show the predictions with confi- 
dence intervals using neural networks. The BNN-ZAE model 
reproduces the overall trend for the data of 3!P and As and 
slightly overestimates the cross sections of 3!P in the most 
energy region. However, it neglects the two obvious peaks 
for 165Ho and predicts the cross sections with only one peak. 
At energy £ = 14 MeV, where the data are approximately 0.25 
b, the BNN-ZAE model grossly overestimates the cross sec- 
tion. This process is illustrated in Fig. 3. The BNN-OPT 
model provides a smaller RMS deviation than the BNN-ZAE 
model. This point is reiterated by comparing the predictions 
of the BNN-OPT and BNN-ZAE models. However, the confi- 
dence intervals provided by the BNN-OPT model were wider 
than those provided by the BNN-ZAE model. When extrap- 
olating the cross section to the low-energy region, € < 10 
MeV for 31P and As, overfitting by the BNN-OPT model 
is shown. The confidence intervals show an increasing cross 
section with decreasing energy, which was not observed in 
the experiment. 

The LBNN model reproduced the data better than the 
other two models did. Two Lorentzian shapes related to the 
quadrupole deformation parameters (2 were considered in the 
LBNN model, as shown in Fig. 1(c), hence the predictions 
show two peaks for deformed nuclei. It cannot be denied 
that the LBNN model underestimates the cross section of 3!P 
at € = 21.5 MeV, where the calculations show a valley be- 
tween two Lorentzian peaks, but the data display a weak peak. 
The weak peak may reveal a substructure beyond the main 
Lorentzian shapes, which can also be observed in the data 
of 2°°Pb near € = 25 MeV. However, its origin has not been 
explained by the physics-motivated model, and hence, it has 
not been considered in the neural network. The improvement 
from BNN-OPT to LBNN supports the idea that the substruc- 
ture beyond the main Lorentzian shapes can be considered in 
the neural network after its properties have been revealed. 


IV. CONCLUSION 


In conclusion, the photoneutron yield cross sections as a 
function of the charge number Z, mass number A, and in- 
cident energy £ were studied using the BNN, and the model 
is abbreviated as BNN-ZAE. The numerical parameters of the 
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Fig. 4. (Color online) Photoneutron yield cross sections for spherical nuclei °Zr, ‘'*Sn, and ?°°Pb as a function of the incident energy. The 
circles with error bars show the experimental data taken from the EXFOR database. The curves with shadows show the predictions of the 
three types of Bayesian neural networks (as shown in Fig. 1). 
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Fig. 5. (Color online) Same as Fig. 4 but for the deformed nuclei *"P, As, and '®°Ho. The quadrupole deformation parameters 82 of those 
nuclei are shown. 


neural network were varied to test the model. The influence of the activation function on the prediction was determined. The 


sigmoid activation function was the best for realizing nonlin- 
earity between the input and output, and hence it provided 
the smallest deviations between predictions and data. How- 
ever, the predictions of the BNN-ZAE model could not be 
improved by increasing the number of hidden layers from 1 
to 3 and the number of hidden nodes from 10 to 100. 

In the method proposed in Ref. [38], physics guidance is 
provided in BNNs from the input layer. Several effects of 
the photoneutron reaction, such as the isospin dependence 
and shape effect, have been observed [3, 39, 40], which in- 
dicate that the cross sections of the photoneutron reaction de- 
pend on the ground-state properties of the nuclei. Based on 
this knowledge, the optimal ground-state properties were se- 
lected as input neurons, resulting in the BNN-OPT model. It 
was shown that the deviations between the predictions of the 
BNN-OPT model and the data were smaller than those of the 
BNN-ZAE model. 

The BNN was further improved by the Lorentzian shape of 
the photoneutron yield cross sections. The Lorentzian func- 
tion was applied to map the hidden nodes to the output cross 
sections, and the empirical formula of the Lorentzian param- 
eters was applied to link the input nodes to the output cross 
sections. This new algorithm is called Lorentzian function- 
based BNN (LBNN). We evaluated the BNN-ZAE, BNN- 
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